Detecting Comparative Sentiment Expressions - A Case Study in Annotation Design Decisions

A common way to express sentiment about some product is by comparing it to a different product. The anchor for the comparison is a comparative predicate like “better”. In this work we concentrate on the annotation of multiword predicates like “more powerful”. In the single-token-based approaches which are mostly used for the automatic detection of comparisons, one of the words has to be selected as the comparative predicate. In our first experiment, we investigate the influence of this decision on the classification performance of a machine learning system and show that annotating the modifier gives better results. In the annotation conventions adopted in standard datasets for sentiment analysis, the modified adjective is annotated as the aspect of the comparison. We discuss problems with this type of annotation and propose the introduction of an additional argument type which solves the problems. In our second experiment we show that there is only a small drop in performance when adding this new argument type. 1